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Abstract 

Network of nonlinear dynamical elements often show clustering 
of synchronization by chaotic instability. Relevance of the clustering 
to ecological, immune, neural, and cellular networks is discussed, with 
the emphasis of partially ordered states with chaotic itinerancy. First, 
clustering with bit structures in a hypercubic lattice is studied. Spon- 
taneous formation and destruction of relevant bits are found, which 
give self-organizing, and chaotic genetic algorithms. When sponta- 
neous changes of effective couplings are introduced, chaotic itinerancy 
of clusterings is widely seen through a feedback mechanism, which sup- 
ports dynamic stability allowing for complexity and diversity, known 
as homeochaos. Second, synaptic dynamics of couplings is studied 
in relation with neural dynamics. The clustering structure is formed 
with a balance between external inputs and internal dynamics. Last, 
an extension allowing for the growth of the number of elements is 
given, in connection with cell differentiation. Effective time sharing 
system of resources is formed in partially ordered states. 

1 Dynamical Viewpoints 

As is discussed in the preface, dynamical viewpoints have been appreciated in 
biological sciences. Here we need some logic to understand complex dynam- 
ical networks. Such studies are required from neural, immune, cellular, and 
ecological networks. In neural systems, Tsuda has stressed the importance 
of (chaotic) dynamics in functions over several years ^ ^, ^ . Freeman has 
noticed the importance of the change of the degree of coherence of neural 
activities In the epilepsy, an ensemble of neurons exhibits a large spike 
due to the coherent oscillation of neural activities. Partial synchronization of 
nonlinear oscillations has been discovered in the visual cortex of a cat [§, 0]. 
Vaadia and Aertsen ^ have found that the effective coupling among neu- 
rons varies temporally in a rather short time scale. They have found that the 
degree of synchronization of oscillations change both temporally and by the 
choice of pairs. In an ecological system, many species coexist in a network 
of food web. The population dynamics of species seems to be more stable 
as the complexity of network is larger, as Elton has discovered in the forest 
of England |jlO|. Furthermore the stability may not be sustained as a fixed 
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small population. Temporal variation of populations there is so large that the 
diversity in rain forests is often believed to be maintained only in a nonequi- 
librium state . Similar interacting population dynamics is also important 
in the immune network, where Jerne proposed the network of antigens and 



antibodies |T^. Possibility of many attractors in such network system is 



discussed [14] in relation with spin glass type models |15], while temporally 
successive switches of many states are discussed in |TB|. At a somatic level, 
metabolic reactions often show nonlinear oscillations through catalytic reac- 
tions. In a developmental process and cell differentiation, interaction among 
cells is important besides the control by gene switches. Creation of diverse 
cells by the latter mechanism is often discussed in relation with many fixed 
point attractors fl^, while dynamical viewpoints in cellular interactions are 
stressed in a recent experiment [p!8| , |T9|. In these fields, studies on dynamic 
nonlinear networks are strongly requested, where a huge number of interact- 
ing nonlinear elements is involved. So far, spin-glass type models are used as 
a standard one for a system with many fixed-point attractors organized as a 
tree structure. With such models, static aspects or relaxation towards sta- 
tionary states are studied. To address the dynamical problems listed above, 
however, we need studies of a system with many nonlinear interacting ele- 
ments. The purpose of the present paper is to point out that the network 
of chaotic elements can provide a novel standard framework for a variety of 
biological networks with dynamical complexity. The thesis of the present 
paper is motivated by the previous studies by the author on an ensemble 



of chaotic elements. There [EG, 21 1 it was found that clustering of synchro- 



nization is a general and important feature in globally coupled dynamical 
systems. Elements split into few or many clusters, in which their oscillations 
are synchronized. The number of clusters can differ by attractors, and by the 
strength of chaos. Complex partition into clusters is also found. This com- 
plexity is partly common with the spin glass type problems ||22|. Generally 
speaking, there are three possibilities in clustering; phase, amplitude, and 
frequency of oscillations. For example, in the pure phase clustering, the am- 
plitudes and periods of oscillations of elements are identical; only the phases 
of oscillations differ by clusters to which elements belong. So far the cluster- 
ing we have studied does not purely consist of only one of the above three 
types. Phase, amplitude, and frequency clusterings are mixed, although the 
phase difference is most relevant to clusterings. In neural systems, oscilla- 
tions seem to split into clusters, as discussed as "gravitational clustering" by 
Vaadia and Aertsen These clusterings are rather complicated, although 
the phase differences seem to be most important. In clustering it should be 
noted that identical chaotic elements differentiate spontaneously into differ- 
ent groups: Even if a system consists of identical elements, they split into 
groups with different phases of oscillations. Hence a network of chaotic ele- 
ments gives a theoretical basis for differentiation of identical elements, and 
provides a mechanism on the origin of diversity and complexity in biological 
networks. Besides this complexity in fixed relationships ( since two elements 
in the same cluster remain at the same cluster there), dynamical changes of 
relationships and synchronizations are of importance in the biological prob- 
lems listed above. Indeed, the network of chaotic elements shows dynamical 
complexity, when chaotic instability in each element is stronger; a typical 
example here is chaotic itinerancv 120, EBl 131, EUfsee also ^2). The mainte- 



problem in an evolutionary system. A dynamical mechanism of maintenance 
of diversity is recently proposed as liomeochaos ^ . Here we also discuss 



a possible relationship between homeochaos and clustering. The purpose of 
the present paper is to survey the relevance of the idea of clustering to biolog- 
ical systems. For the application of the idea, it is often necessary to extend 
the basic network of chaotic elements to different topology, to non-uniform 
elements, to synaptic couplings, and to systems with variable degrees of free- 
dom. In the next section we give a brief review of basic results of clustering, 
and chaotic itinerancy in globally coupled maps (GCM). In §3, the cluster- 
ing idea is extended to a system with hypercubic topology. This extension is 
motivated by interacting population dynamics with mutation and its appli- 
cation to genetic algorithms. The formation of synchronized clusters strongly 
reflects the bit structure in the lattice. Indeed we will see self-organization 
and destruction of relevant bits and "don't care" bits by the chaotic itiner- 
ancy mechanism. In §4, we further extend the system in §3 to allow for a 
change of the coupling strength. The motivation for this extension comes 
from a system with interacting populations with mutation of mutation rates. 
By the last process, the coupling strength among elements (species) is effec- 
tively changed with time. It turns out that the system attains a dynamic 
stability allowing for diversity of many groups, by forming a feedback mech- 
anism to adjust the coupling strength. This mechanism, called homeochaos 
turns out to be sustained by successive changes of clusterings. §5 and §6 
are devoted to extensions of our GCM to synaptic coupling cases, moti- 
vated by applications to neural systems. In §5, a globally coupled map with 
distributed coupling strengths is shown. Different types of clustering behav- 
iors, from synchronized to completely desynchronized, are observed within 
a unique system. This observation opens up the possibility of controlling 
the degree of synchronization of elements according to inputs, by modifying 
the coupling strengths among the elements. Such control is carried out by 
a synaptic model introduced in §6, which is in a possible relationship with 
the synchronization by external inputs in the brain [§. In §7, we study clus- 
terings in a system with growing degrees of freedom, in connection with cell 
differentiation and growth. It is found that the dynamic clustering leads to 
growth of the number of cells by forming a time sharing system of foods 
(resources). §8 is devoted to a brief summary and discussions. A rather 
personal view on biological networks from the above lines is given in Table I. 
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2 Brief Review of Globally Coupled Maps 



The simpliest case of global interaction is studied as the "globally coupled 
map" (GCM) of chaotic elements An example is given by 



e ^ 

= (1 - e)f{x^it)) + - ^ (1) 

where n is a discrete time step and i is the index of an element [i = 1,2, - ■ ■ ,N 
= system size), and f{x) = l — ax^. The model is a mean-field-theory- type 
extension of coupled map lattices (CML) |]2^. The above dynamics consists 
of parallel nonlinear transformation and a feedback from the "mean-field". 
It is equivalent to i/„+i(i) = /[(I - e)yn{i) + with the aids of 

transformation t/nii) = f{xn{i))- In this form, one can see clear correspon- 
dence with neural nets: If one chooses a sigmoid function {e.g.,tanh{i3x)) as 
f{x) and a random or coded coupling eij, a typical neural net is obtained. 
Through the interaction, some elements oscillate synchronously, while chaotic 
instability gives a tendency of destruction of the coherence. Attractors in 
GCM are classified by the number of synchronized clusters k and the num- 
ber of elements for each cluster N^. Here a cluster is defined as the set of 
elements in which x{i) = x{j) pSf. Each attractor is coded by the cluster- 
ing condition [k, {Ni, N2, ■ ■ ■ , Nk)]. If we distinguish each element i, there 
are A^!/(A'^i!Ai"2! ■ • ■ Nk\) ways of the partitions for each clustering condition 
(A^i, N2, - ■ ■ , Nk). We have exponentially many attractors per each clustering 
condition. An interesting possibility in the clustering is that it provides a 
source for diversity. Even if the system is started from identical states, they 
split into different groups. In §7, we discuss the possibility of a role of clus- 
tering in cell differentiation. In a globally coupled chaotic system in general, 
the following phases appear successively with the increase of nonlinearity in 
the system (a in the above logistic map case) pO[: (i) Coherent phase: A 
coherent attractor {k = 1) has occupied (almost) all basin volumes. 

(ii) Ordered phase: Attractors {k = o{N)) with few clusters have oc- 
cupied (almost) all basin volumes. 

(iii) Partially ordered phase: Coexistence of attractors with many 
clusters {k = 0{N)) and few clusters. 

(iv) Turbulent phase: All attractors have many {k = 0{N); in most 
cases k ~ A^) clusters. 

In the turbulent phase, although x{i) takes almost random values almost 
independently, there remains some coherence among elements. Indeed the 
distribution of the mean field h = {l/N)J2jf{x{j)) does not obey the law 
of large numbers. The emergence of hidden coherence is a general property 
in a globally coupled chaotic system |]2T|. Existence of such coherence may 



be important to discuss about the EEC In EEC, one measures a given av- 
erage of neuronal (electric) activities. Since a firing pattern of each neuron 
is not regular ( i.e., chaotic or random), the amplitude of the variation of 
EEC might decrease with the number of neurons involved in the average. 
Still, we have observed a large enough amplitude of variation in EEC This 
observation suggests that there remains some correlation among neuronal 
bursts. If each neuronal bursting were random, it would be hard to imagine 
a mechanism to keep such coherency. The above hidden coherence in glob- 



There are a variety of attractors with a different number of clusters, and a 
different way of partitions [Ni, N2, - ■ ■ , N^]. We have measured the fluctua- 
tion of the partitions, using the probability Y that two elements fall on the 
same cluster. This Y value fluctuates by initial conditions. This fluctuation 
seems to remain finite even if the size goes to infinity ||22|. Such remnance 



of fluctuation of partitions is also noted in spin glass models [^. In the 
PO phase, this fluctuation is enhanced. This increase of complexity at the 
partially ordered state may be important to study the relevance of the PO 
phase to biological systems. We also note that the partition is usually inho- 
mogeneous, and is organized as an inhomogeneous tree structure as in the 
spin glass model |l^ . Besides the above static complexity, there emerges 
dynamic complexity in our model at the PO phase. The orbits make itiner- 
ance over ordered states via highly chaotic states. In the ordered states the 
motion is partially coherent. Our system exhibits intermittent changes 
between the self-organization towards the coherent structure and 
its collapse to a high-dimensional disordered motion. This dynam- 
ics, called chaotic itinerancy, has been found in a model of neural dynamics 
by Tsuda optical turbulence and in GCM. Here a number of ruins 



of low-dimensional attractors coexist in the phase space. The total dynam- 
ics consists of the residence at a ruin and a high- dimensional chaotic state 
interspersed between the residence. 



3 Bit Clustering in Hypercubic Coupled Maps: 
Basis of Chaotic Genetic Algorithm 

Let us discuss some (population) dynamics of many individuals, coded by 
genes. If genes are represented by a bit sequence, the mutation process in 
gene space is given by the diffusion in the bit sequence. When some nonlinear 
population dynamics is included to take into account of the saturation, com- 
petition, or prey-predator (host-parasite) interaction, the total population 
dynamics is given by the local nonlinear dynamics and the diffusion process 
on the hypercubic lattice of length 2, corresponding to the bit sequences. 
The minimal model for this process is given by the following coupled map on 
a hypercubic lattice; 

x„+i(z) = (1 - e)/(x„(^)) + f: /(x„(a,(^))), (2) 

^ i=i 

where (Jj{i) is a "species" whose j'th bit is different from the specie i (with 
only one bit difference) , and K is the total bit length of species ( the number 
of total "species" is 2^). We use a decimal representation of bit sequence 
often; for example 42 stands for the sequence 101010, and (72(42) = 40 The 



present model may be relevant to genetic algorithms [^], where population of 
bit-strings changes according to their fitness. The model is also of theoretical 
interest, since it lies between globally coupled and locally coupled models 
(CML) In a global coupled chaotic system, we have N connections per 
element, while a d-dimensional CML (with nearest neigbor coupling) has 
2d = o{N) connections per elements. In our hypercubic system with N = 2^ 
elements, we have K — loooN connections Der elements HSOl. In the model 



nonlinearity parameter a is not large. Here a synchronized cluster means, as 
in §2, that two elements in the cluster oscillate in complete synchronization, 
(j) for two elements i and j in the cluster. Here the split to two 
clusters is organized according to the hypercubic structure. For examples, 
following types of clusterings are observed. (A) 2 clusters by 1 bit: Elements 
split into two synchronized clusters. All elements in each cluster oscillate in 
synchronization as shown in Fig. la) and b). In Fig. 1, successive snapshots 
of Xn{i) are plotted as a function of i. Such clustering into two is frequently 
observed in a globally coupled map system, where the partition into two 
clusters is arbitrary. In our HCM, the partition is governed by the spatial 
structure in the hypercubic lattice. For example, elements may be grouped 
into two clusters with **0*** and (* means that the symbol there 

is either one or zero), each of which has 2^~^ species. If the elements split 
into the group of ******! and ******o for example, the snapshot of x{i) 
shows a zigzag structure ( with period 2), if plotted as a function of the 
decimal representation i (see Fig. la), while the split by the K-th bit leads to 
a periodic structure with period 2^. This clustering is formed by cutting the 
K-dimensional hypercube by a hyperplane (see Fig. lb). (B) 2 bit clustering 
Depending on initial conditions and parameters, the number of relevant bits 
for clustering can be larger than the case (A), as well as the number of 
clusters. In the 2 bit clustering there are following possibilities (BO) 4-cluster 
state with two relevant bits: This case is just a direct product of the previous 
case (A). The hypercubic space splits by two hyperplanes. Elements split 
into four clusters, for example, coded by gi*****, ig*****, j^-j^*****^ 
00*****. (Bl) 2-cluster with 2 bits (XOR construction) We have also found 
an attractor with 2 clusters with the use of 2 relevant bits. For example the 
elements split into the groups (i) IQ***** or Ql***** and (ii) QO***** or 
11*****. This split corresponds to the construction of XOR (exclusive or) 
with the use of 2 relevant bits (see FIg.lc). (B2) 3-cluster with 2 bits We have 
also found a 3-cluster state with 2 bits, constructed for example as (i)io***** 
Q^***** (^jj^) g^n^ (^jjj^ 00*****. Here we have to note that not all 

partitions are possible in the HCM. Even if we start from an initial condition 
with a given clustering condition, the synchronization condition {x{i) = x{j) 
for i,j belonging to a same cluster) is not satisfied at the next step, for most 
of such initial conditions. In contrast with the GCM case, not all possible 
partitions can be a ( stable or unstable) solution of the evolution equation. 
One can easily check that the synchronization is preserved for the clustering 
(A), (BO), (Bl), and (B2). Generally the clustering should be constructed as 
a combination of hyperplane cuts, and the condition of a cluster is written as 
a bit representation with the symbol "*" , corresponding to the "don't care" 
bit in genetic algorithms [0. (C) 3 bit and higher (K) bit coding: 

Clustering with the use of bits more than 2 are constructed in a similar 
manner. Most clusterings observed here are direct products of (A) or (Bl)- 
(3). (CI) parity check: 2 clusters from K bits; elements split into two groups 
according to the parity of the number of I's in each bt representation. For 
example, elements split into two clusters as follows: (i) 000, Oil, 101, 110 
and (ii) 001 010, 100, 111, for K = 3. The clustering, thus gives a parity 
check. It is a hypercubic version of the zigzag (1-dim) or checkerboard (2- 
dim) pattern (C2) Hamming distance code : K + 1 clusters from K 

bits; elements spht into K + 1 clusters according to the HamminE distance 



For example, elements split into 4 clusters as follows (i) 000 (1 element) , 
(ii) 001 010, 100 (3 elements) , (iii) 011,101, 110, (3 elements) and (iv) 111 
(1 element) , for = 3. This clustering is constructed by the cuts by K 
parallel hyperplanes. An attractor with many clusters is often constructed by 
a direct product of combinations of (AO), (CI), and (C2) , that is of (0) 1 bit 
code by a hyperplane cut (1) Parity check and (2) Hamming distance cuts. ( 
see for example Fig. Id) for a 6-cluster attractor). If local chaos is stronger 
(a is larger), the clustering is not complete: Some elements stay very close 
and oscillate almost synchronously over some time steps, but then they are 
separated due to chaotic instability. At some medium nonlinearity regime, we 
have found a chaotic itinerancy state, where relevant bits change according 
to temporal evolution. In Fig. 2, change of relevant bits for the clustering is 
seen. So far this state is seen only in a very narrow parameter regime, and is 
found as very long transients before the system finally falls on an attractor 
with few clusters. With the introduction of external inputs to each element, 
it is also possible to have a clustered state following the external information 
[ p^ . Relevant information is extracted through this process spontaneously, 
which is stored as a relevant bit in the clustering. Fig. 1 Fig. 2 



4 Homeochaos and Clustering: Dynamic Main- 
tenance of Diversity 

In a system with interacting population dynamics, it is an interesting ques- 
tion how diversity of genes are maintained. In ||2^, we have proposed a con- 
cept "homeochaos" as a mechanism for sustaining dynamic stability with di- 
versity. There, population dynamics models with interaction among species. 



mutation, and mutation of mutation rates p5| , pq] has been studied. In 
particular, we take a 2-dimensional map for local dynamics here, since we 
are interested in the interaction between hosts and parasites (or preys and 
predators). Each individual has a gene coded by a bit sequence as in §3. 
We assume that the parasite can attack a host only if their bit strings are 
completely matched. By this restriction, we can have a 2-dimensional map 
for the local population dynamics of each bit string. To be specific we have 
studied the model 

h\i) = a(l - h{i)){h{i))exp{-l3p{i)) (3) 

p\j) = h{i){l - exp{-Pp{i))) (4) 

instead of the 1 dimensional logistic map in §3. Here the set of variable 
{h{i),p{i)) gives the population of the host and parasite of the "species" i. 
The term exp{—(3p{i)) represents the fraction that is killed by the correspond- 
ing parasite i. Further we assume that each "species" i can have different 
mutation rates, in other words, each is coded by rather than i. Each 
group has a mutation change from its nearest neighbor point in the hy- 
percubic lattice a{i, k) in §3 with the rate given by the mutation level j (here 
we assume ej = 2*^-^""'^~-^^/^ and jmax = 30). Since the mutation level does 
not affect the interaction, the local dynamics is obtained straightforwardly 



the sum of the population of each "species" over all mutation levels. Then 
the local dynamics of h''{i) and p^{i) obey exactly (3) and (4). The local 
dynamics h{i,j) h'{i,j) is given by multiplying the dynamics of h'^{i) by 
hn{i, j)/hf^{i), the fraction of the population of the level j. (Of course the 
dynamics p{i, j) p'{i, j) is given by multiplying the corresponding equation 
by Pn{'i,j)/Pn{'i)-) Thus the total dynamics, (which is a little bit complicated 
but is straightforwardly obtained ) is given by the coupled map lattice with 
the diffusive coupling by the mutation. The coupling strength ej depends on 
the level j, as stated, and there is also diffusive couphng between neighboring 
mutation levels j — j ± 1, with the (mutation ) rate ( see [0]). Through 
the mutation of mutation rate, the mutation level is sustained at a high level, 
with some temporal fluctuation. As is studied in p6[|, weak high-dimensional 
chaos is found in this case, which affords the stability with diversity. Such 
stability, called "homeochaos" is proposed to be essential to the stability of 
biological networks In Fig. 3, we have plotted snapshots of h'^{i) suc- 

cessively, where changes of bit clustering are found with chaotic itinerancy. 
In Fig. 4a), we have also plotted the time series of the effective degrees of 
freedom, defined as the number of clusters with a given (finite) precision; 
i.e., h^{i) and h'^{j) agree within the precision, they are assumed to be the 
same "effective" cluster. Drops of the effective degrees give the emergence 
of "almost" clustered states as in Fig. 3a). Corresponding time series of the 
average mutation level is given in Fig. 4b). This type of chaotic itinerancy is 
seen in a narrow parameter regime in the previous section ( e.g. only around 
a ~ 1.52 in the model in §3 for e ~ .3), as long transients. In the above 
two variable model, it is seen only around a ~ 3.75 (as long transients), 
when the mutation rate is fixed ( e.g., at j = 25). On the other hand, such 
behavior is seen over all regimes for a > 3.5, with the inclusion of mutation 
of mutation rates. Hence there must be a mechanism to adjust the effective 
coupling strength so that ( i.e., the mutation rate here) the system stays 
around at the partially ordered state. Such adjustment mechanism is impor- 
tant in homeochaos. A high mutation level is necessary to have a state with 
few clusters. As the mutation level is increased, the oscillation tends to be 
synchronized ( recall that the number of clusters in coupled maps decreases 
with the increase of coupling strength; see §2 and §3). By examining the 
change of mutation rates (coupling strengths) and the change of clustering 
in detail, we have found the following feedback mechanism to keep the sys- 
tem at the partially ordered state: (i) increase of mutation level leads to the 
synchronization of population oscillation, since the coupling is increased : 
(ii) the synchronization leads to decrease of the mutation level [^: (iii) the 
decrease of mutation level leads to the split of synchronized clusters, since 
the coupling is decreased ( see §2 and §3). (iv) split of clusters is associated 
with the irregular temporal dynamics, which leads to increase of mutation 
level, and then to (i) This feedback mechanism is necessary to maintain 
our system at the partially ordered state with the chaotic itinerancy of bit 
clusterings. Since such partially synchronized state is thought to be impor- 
tant in biological networks 0, P], the above mechanism for evolving to, and 
maintaining, the state is important. It is important to search for a similar 
feedback mechanism in other biological networks. Fig. 3 Snapshot showing 



5 Extensions to Coupling forms with more 
complicated structure 



In a biological system, the elements are not homogeneous. It is often neces- 
sary, then, to assume that local dynamics or a coupling strength depends on 
elements. Also time dependence of such parameters may be necessary. Here 
we extend the previous GCM by allowing for inhomogeneous parameters by 
elements, instead of identical parameters for all. There are two possibilities 
here; choice of distributed a{i) instead of a constant nonlinearity a or dis- 
tributed e{i) instead of a constant coupling e. For the latter case, the model 
is given by 

e(i) ^ 

= (1 - e{^))f{x^{^)) + ^ E fMj))- (5) 

In Fig. 5 we have plotted a snapshot of for a system with homogeneously 
distributed coupling over [€min,emax] (i-e., e{i) = e„j„ + {smax - emm)]^)- As 
is seen, the clustering structure depends on the coupling strength at each 
element. Elements with large coupling ( e > .22) form a single synchronized 
cluster, while the number of clusters increases successively with the decrease 
of the coupling strength at the element. Such clustering bifurcation looks 
similar with the phase change i) iv) in §2. However, this is not a trivial 
extension. The clustering change in §2 is the bifurcation with system param- 
eters, while the change here is included in a a single network system, where 
all the elements therein are connected by a unique mean field. Still the inter- 
nal bifurcation among elements occurs here. At the edge parameter region 
between clustered and turbulent states the motion is rather complicated with 
chaotic itinerancy. Desynchronized bursts emitted from the elements with 
smaller e{i) flow to elements with larger e{i), where clustering can change 
in time. The behavior discussed here is also seen in a model with the dis- 
tributed nonlinearity, i.e., a{i) = amin + (amax — cimin)jf- If the range of 
distribution of coupling is small, the behavior there approaches that in the 
previous section. According to the width of the coupling range variety of 
clusterings decreases. We note that the hidden coherence still emerges in the 



turbulent state, even if the parameters a or e are distributed ||2T|. Fig. 5 x(i) 
vs i a) Snapshot b) Overlaid 



6 Synaptic Dynamics 

In traditional neural networks, coding is assigned at firing patterns x{i), 
while the synaptic change and memory are assumed to be assigned on the 
interaction strength between two elements i and j. Recently there are some 
arguments that dynamical coding, given by the correlation between x{i) and 
may be important ||^, Then it may be interesting to pursue this 
"converse" limit: In other words, let us assume that the synaptic change and 
memory are assigned on the coupling e(i) at i, not on the interaction between 
i and j. Thus we take the following model 



where e„(i) is in-(de-)creased according to inputs on the i-th element. To be 
specific, we take the following dynamics; first increase the coupling to the 
mean field, according to the input s„(i) by 

e'{i) = e„(i) + jSn{i), (7) 

and then rescale the coupling so that the average of e(z) is conserved: 

,., e'(i) 
^n.iW = eoX^-^. (8) 

Indeed the latter equation is introduced as a suppression, and may be re- 
placed by other forms to suppress the overgrowth of e„(z). What we need here 
is a mechanism for (a) the suppression of an indefinite increase of coupling 
e{i) and (b) the competition among elements for the increase of coupling. 
When common inputs are applied to elements io < i < ii-, synchronization 
degree of the oscillations Xn{i) for < i < ii increases. Depending on the 
input strength, two elements i, j with common inputs often synchronize com- 
pletely. After inputs are eliminated, a pair of (almost) synchronized elements 
remains coherent or correlated. We have thus achieved clustering according 
to inputs This clustering is often preserved even if the couphng e„(i) 



is again restored to be homogeneous. Generally, the clustering is formed 
according to the structure of inputs. We have made some simulations with 
correlated inputs in p groups per N/p elements. In particular we take 

Sn{i) = Rn{tnt{^)) + r„(z) (9) 

where Rn is a random signal with the amplitude 1 but common for each group 
k- <i< (k + l)^ while r„ is a noise depending on elements but with much 
smaller amplitude 6 {6 < 1). Adopting this form, inputs are random, but 
are strongly correlated within each group of N/p elements. In Fig.6 we have 
plotted the spacetime diagram when these inputs are applied. Correlated 
motion within each N/p elements is seen. Some of the correlation remain 
even after the inputs are eliminated ( see Fig. 6b)). To see the correlation in 
oscillations {xn{i)} more quantitatively, we introduce the difference matrix 

Aij =< (x„(j) - Xn(i) f >, (10) 

where < ... > is temporal average. The difference matrix is plotted in Fig. 7 
for the inputs in the above structure. We note that the partial clustering is 
formed according to the structure of inputs as long as the number of input 
groups is small. This organization is not trivial since we have not imposed 
any direct change of the coupling to increase the coherence among elements 
with correlated inputs. No enhancement of the connection between pairs 
is required here. Indeed, given an element, the coupling strength takes a 
same value for any pair between it and other elements. The formation of 
clustering according to inputs works well if the parameters a and eo are 
chosen so that the number of clusters there is not far from the expected 
cluster number by inputs. If the number of clusters at the corresponding 
a and eo values is smaller than that of the inputs, some of the groups are 



by the balance between internal dynamics and the external inputs. In Fig. 7 
a), for example, 3 out of 4 input groups are mapped into the clustering, 
but the other group is not clearly mapped. In Fig. 7b), two of the four 
input groups are mapped, while the other two split into smaller clusters. 
Fig. 6 spacetime diagram Fig. 7 difference matrix There is a recent report 



of an interesting experiment by Hayashi |Q, where synchronization among 
neurons is increased when a rat is continuing a task with a feedback to brain. 
In our model this corresponds to the growth of e{i) for all elements. With 
further continuing the task, the synchronization reaches maximum, and the 
rat's brain goes to an epileptic state, where the information processing ability 
is lost. To avoid such catastrophe, we have to introduce some mechanisms to 
suppress the increase of e in our model. We may expect that such mechanism 
exists in the real brain, and that its breakdown leads to the epilepsy. 



7 Growing Coupled Maps: Origin of Diver- 
sity and Differentiation 

One missing feature in our network of chaotic elements so far, is the pos- 
sibility to change the degrees of freedom themselves. In biological systems, 
often the number of elements itself varies. Such growth of elements is also 
seen in the economics. Here we study a very simple model with the growth 
of the number of elements, taking a coupled map model. A related but more 
realistic model for cell division and differentiation is given in |T^. We as- 



sume that there is a variable x{i) determining the cell state, and that cells 
compete with each other for a source term s. Source term s is supplied from 
outer environments with a constant rate c. The ability to get this source 
depends on the internal state x{i), with some nonlinear function f{x). Thus 
the dynamics of each x{i) is given by 

Xn+l{i) = Xn{i) + f{Xn{i)) - Sn] (H) 

The term a;„(i) = f{xn{i))—Sn gives a source term that the element i 

takes at the time step n. The second condition assures J2i{xn+i{i) — Xn{i)} = 
c, that is, the sum of the source term balances with that supplied externally. 
Further we introduce the following dynamics for cell division and death. 
(A) Divide the cell i if > Tg] After the division, x of the cell i and 

the new element + 1 is assigned to be — Tg)/2 + 6 and — 

Tg)/2 — 6, with a very small random number 6. (B) Remove the cell i 
if Xn{i) < Td] Here we fix = 1 and = 0, although our results are 
essentially independent of the choice of Tg and T^. The function f{x) is 
chosen to be f{x) = ^sin{2nx) Time series of the number of cells and 
the effective number of clusters are plotted in Fig. 8, while the maximal and 
average numbers of cells are plotted as a function of K in Fig. 9. We note 
that the growth is maximal around K ^ 4.3. In the corresponding coupled 
map model with a fixed number of elements (i.e., globally coupled circle map 



2 < K < 4.1, at the partially ordered phase around 4.1 < K < 4.4, and at 
the turbulent phase for K > 4.4. Thus our result suggests that the growth 
is enhanced at the partially ordered state. If oscillations of all elements 
are synchronized, they compete for the source term at the same timing. 
This hard competition is not good for an effective use of resources. By the 
clustering, a sort of time sharing system is constructed. Thus resources are 
effectively used with some oredering by elements. On the other hand, if the 
elements are completely desynchronized, no ordering for the use of resources 
is possible. In this case effective use of resources is again impossible. Thus the 
growth is enhanced at the partially ordered state, where synchronization is 
lost but there remains some ordering. At the partially ordered state, typical 
clustering is quite inhomogeneous. Some elements form a large cluster, while 
there are many other elements with desynchronization. Cells belonging to 
a large synchronized cluster grow much slower than desynchronized cells. 
When these synchronized cells divide, the number of cells suddenly increases 
by A^^i, the number of elements in the cluster. This increase causes a hard 
competition for resources, and often leads to spontaneous death of many 
cells. Such multiple death reminds us of the programmed death known in real 
biological developmental processes. Our growth and division model may also 
have relevance to economics, where many individuals or companies compete 
for finite resources. Growth, division, death (bankrupt) factors are important 
in economics. Time sharing for resources is useful there. Economic crash 
may be related with the above multiple cell deaths, and may be due to 
synchronized behaviors of agents for resources. The present model, of course, 
is too simple to the study of cell growth and differentiation. Here we have 
discussed this model as the simplest illustration. For a model including 
metabolic reaction, and other stimulating results, see |^ in the present 
proceeding. In the present model we have not found a state corresponding 
to the stage III in Possibly we need at least a model with two variables, 
phase and amplitude, to have the stage III where the separation of poor and 
rice cells emerges. Fig. 8 



8 Summary and discussions 

In the present paper we have discussed relevance of dynamic clustering to 
biological problems. In a network of chaotic elements, they often split into 
synchronized clusters due to chaotic instability. Identical elements spon- 
taneously differentiate. Thus the clustering can give a basic concept for 
the origin of diversity. Indeed we have applied the clustering mechanism to 
cell differentiation in §7. By using a hypercubic topology and local nonlin- 
ear dynamics, we have studied clusterings with bit structures. Clusters are 
spontaneously formed reflecting the bit structure. Relevant bits are spon- 
taneously formed, which opens up the possibility of self-organizing genetic 
algorithms. Since viruses form quasispecies coded in a hypercubic space 
as Eigen et al. discusses it may be interesting to search for dynamic 



clustering there. When chaotic instability and averaging by couplings are 
somewhat balanced, a partition into clusters is very complex. In this par- 
tially ordered state, dynamics is also complex with chaotic itinerancy over 



ancy to biological networks have been discussed throughout the paper. In 
the hypercubic topology, chaotic switches of relevant bits are formed succes- 
sively. Besides possible relevance to dynamics of immune networks and virus 
populations, such chaotic itinerancy of bits may be of use in genetic algo- 
rithms. Relevance of the partially ordered states to ecological and immune 
networks is studied with the inclusion of the change of effective couplings 
as mutation of mutation rates. It turns out that the system maintains its 
stability as homeochaos, by forming a feedback mechanism to keep the sys- 
tem around partially ordered states. The homeochaos provides a mechanism 
of the maintenance of diversity, important in ecological and othe biological 
networks. In a model allowing for the growth of the number of elements, 
we have found that effective time sharing system of resources is formed in 
partially clustered states. This study, originally motivated in the cell differ- 
entiation and division, may be applied to economics, where a breakdown of 
the time sharing system by synchronization may lead to economic crash. In 
a GCM with synaptic couplings, we have observed the clustering formation 
through some interference of external inputs and local dynamics. To have a 
capacity to construct a map of complex environment, it is desirable to have a 
potentiality of complex partitions to clusters, supported by partially ordered 
states. To sum up we have studied classes of extensions of coupled maps to 
hypercubic topology, synaptic couplings, growing degrees, and so on, in order 
to understand the origin and maintenance of diversity and complexity in bi- 
ological networks, acknowledgements 1 am grateful to Ichiro Tsuda , Takashi 
Ikegami, Kenji Araki, Hatsuo Hayashi, Ad Aertsen, Hiroyuki Ito, Hiroshi 
Fujii, Walter Freeman, and Tetsuya Yomo for stimulating discussions. I am 
again grateful to Takashi Ikegami ( for §3 and §4), Kenji Araki ( for §6), 
and Tetsuya Yomo (for §7), for exciting discussions through collaboration on 
related works p5| , p6| , |33| , p!9| . The work is partially supported by Grant- 
in-Aids for Scientific Research from the Ministry of Education, Science, and 
Culture of Japan. 
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Figure Caption Fig. 1 Snapshot x{i) of hypercubic coupled map (2) with 
k = 7 and a = 1.5, plotted as a function of i, decimal representation of the 
bit string, e = .4 for a) and b), e = .3 for c) and d). a) Two successive time 
steps (after 100000 steps) are overlaid for a two-cluster attractor, split by the 
condition ****** and ******!. b) 8 successive time steps are overlaid 
for a two-cluster attractor, split by the condition ***** * and 1 ***** *. 
c) 8 successive time steps (after 100000 steps) are overlaid for a two-cluster 
attractor, split by the XOR condition [* * *0 * 0* or * * *1 * 1*] and [* * *0 * 1* 
or * * *1 * 0*]. (The attractor is a stable cycle with period-4). d) 8 successive 
time steps are overlaid for a 12-cluster attractor. (The attractor is chaotic). 

Fig. 2: Space-time diagram for the coupled map lattice on a hypercubic 
lattice (2) with a — 1.54, e = .3 and k — 6 (i.e., N = 2^). On the correspond- 
ing pixel at a given time and element, a bar with a length proportional to 
{xn{i-ij) — 0.1) is painted if Xn{i,j) > .1. (a) Every 4th time step is plotted 
from 50000 to 52400. (b) Continued from a), but for the time steps 60000 to 
62400. 

Fig. 3 Snapshot plots of hfj^{i) for the model with mutation of mutation 
rates described in the text, a = 3.5, (3 = 7.0, k = 7 and j^ax = 30. (a) time 
step 9000 (b) 18000 (c) 21000 (d) 24000. 

Fig. 4 (a) Time series of the effective degrees of freedom, corresponding to 
the simulation of Fig. 3. An effective cluster is defined as follows; if x{i) and 
x{j) agree within the precision 2~^^. Successive drops of the degrees are seen, 
which corresponding to a bit-clustered state like Fig.3a). (b) Corresponding 
temporal evolution of the average mutation level j. Fig. 5 Snapshot of the 

globally coupled map (5), with a = 1.6, = 10^. Xn{i) at the time step 
n = 5000 is plotted as a function of i. The coupling e(i) is ordered as 
e(i) = 0.4(i/A''). (b) Successive snapshots Xn{i) at the time steps 5001, 
5002, 5003, and 5004 are overlaid. Fig. 6 Space-time diagram for the globally 

coupled map eq. (6) with eqs. (7) and (8) with the inputs eq.(lO), with 
p — A. a — 1.85, 6 — 0.1, 7 = 0.1, N — 128. On the corresponding pixel at a 
given time and element, a bar with a length proportional to {xn{i,j) — 0.1) 
is painted if Xn{i,j) > -l- (a) Every 16th time step is plotted from 8000 
to 16000. (b) Continued from a), after the inputs are stopped at the time 
step 20000. Over the time steps 24000 to 32000. Fig.7 Difference matrix 

eq. (10) averaged over 20000 steps, after inputs of eq. (9) with p = A and 
6 = 0.1 are applied for 20000 steps. The synaptic dynamics (7) and (8) are 
used with eo = 0.3, and 7 = 0.1 (a) N = 16, and a = 1.85 (b) N = 32, 
and a — 1.75. Fig. 8 Temporal evolution of the number of cells N (thick hne) 

and the effective cluster number ( dotted hne) . The effective cluster number 
is defined as that of clusters with the precision 10~^ ( checking the equahty 
between x{i) and x{j) with this precision). Simulations are carried out with 
the use of eq. (11) with c = 0.1, starting from one cell (A^ = 1). (a) K = 2.4 
(b) K = 3.35 (c) K = 3.45 Fig.9 The average ( thick line) and maximal 

number of cells over the time steps 5000 to 15000. Simulations are carried 
out with c = 0.1, and starting from one cell. 



